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Abstract: Based on facial expression (FE), this paper explores the possible use of the affective communication in 
virtual environments (VEs). The attention of affective communication is examined and some research ideas for 
developing affective communication in virtual environments are proposed. We place an emphasis on 
communication between virtual entities, which make use of other data apart from the information being 
communicated. In particular, we explore the capacity of VEs to communicate affective states, i.e. those aspects 
of communication concerned with emotional response, and discover how agents within VEs can model and use 
affective states to enhance the realism of the VEs. Moreover, we discuss the social intelligence that renders 
affective behaviours of software agents and its application to a collaborative learning system. We argue that 
socially appropriate affective behaviours would provide a new dimension for collaborative e-learning systems. 
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1. Problem formulation 

The field of affective computing was proposed and pioneered by Rosalind Picard (Picard 1997) from 
the MIT Media Laboratory. Her definition of affective computing is: “computing that relates to, arises 
from, or deliberately influences emotions.” Her argument for putting emotions or the ability to 
recognise emotions into machines is that neurological studies have indicated that emotions play an 
important role in our decision making process. Our “gut feelings” influence our decisions. Fear helps 
us to survive and to avoid dangerous situations. When we succeed, a feeling of pride might 
encourage us to keep on going and push ourselves even harder to reach even greater goals. Putting 
emotions into machines makes them more human and should improve human-computer 
communication. Also exploiting emotions could lead to a more human decision-making process. 
Consequently, in this paper, collaborative affective e-learning framework aim at reintroducing 
emotional and social context to distance learning while offering a stimulating and integrated 
framework for affective conversation and collaboration. Learners can become actively engaged in 
interaction with the virtual world. Further, the use of avatars with emotionally expressive faces is 
potentially highly beneficial to communication in collaborative virtual environments (CVEs), especially 
when they are used in a distance E-learning context. However, little is known about how or indeed 
whether, emotions can effectively be transmitted through the medium of CVEs. Given this, an avatar 
head model with human-like expressive abilities was built, designed to enrich CVEs affective 
communication. This is the objective of introducing the Emotional Embodied Conversational Agent 
(EECA) (Ben Ammar, Neji and Gouarderes, 2006). We are arguing, then, that the use of peer-to-peer 
network in combination with collaborative learning is the best solution to the e-learning environments. 
Peer-to-peer (p2p) technology is often suggested as a better solution because the architecture of 
peer-to-peer networks and collaborative learning are similar. (Bistrom 2005). This paper explores 
CVEs as an alternative communication technology potentially allowing interlocutors to express 
themselves emotionally in an efficient and effective way. Potential applications for such CVEs 
systems are all areas where people cannot come together physically, but wish to discuss or 
collaborate on certain matters, for example in e-learning, based on the affective communication. 

There are several novel elements to this research. Firstly, although CVEs as a technology have been 
available for more than a decade, user representations are still rudimentary and their potential is not 
well explored, particularly the avatar as a device for social interaction. Secondly, the use of emotions 
to complement and indeed facilitate communication in CVEs is equally under-explored. This is partly 
because early CVEs research was mainly technology driven, leaving aside the social and 
psychological aspects, and partly because the required computing, display and networking resources 
became available only recently. Thirdly, design guidelines for an efficient, effective, emotionally 
expressive avatar for real-time conversation did not exist prior to this research. The multi-agent 
methodology can certainly bring several advantages to the development of e-learning systems since it 
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deals well with applications where such crucial issues (distance, cooperation among different entities 
and integration of different components of software) are found. As a result, multi-agent systems, 
combined with technologies of networking and telecommunications, bring powerful resources to 
develop e-learning systems. In this research work, we propose emotional framework for an intelligent 
emotional system. This system is called EMASPEL (Emotional Multi-Agents System for Peer to peer 
E-Learning), based on a multi-agents architecture. (Ben Ammar, Neji and Gouarderes 2006). 

2. Related works 

Several projects implement learning systems based on multi-agents architectures. Some of them 
work on a generic platform of agents. (Silveira, Bica and Viccari 2000) For example, JTS is a web- 
based environment for learning Java language (Rivera and Greer 2001) based on a CORBA platform 
and using Microsoft agents. In this environment, learners have access to their learner models and 
they are able to change it, in the case they do not agree with the information represented. Another 
example is 1-Help, a web-based application that allows learners to locate human peers and artificial 
resources available in the environment to get help during learning activities. 1-Help is an example of a 
large-scale multi-agent learning environment. (Vassileva, Deters, Greer 2001) Moreover, interesting 
results have been achieved by pedagogical agents (Johnson et al 2000) regarding the learner 
motivation and companion agents acting sometimes as mediators (Conati, Klawe and Socially 2000) 
of the learning process. Finally, tutor agents are usually related to learner modelling and didactic 
decision taking. A virtual learning environment (VLE) provides an environment based on network, and 
the resources in the network are free to share. Therefore, the study process can enhance the 
collaboration among learners. The VLE can help learners do cooperative study, and make necessary 
interactions between each other. For example, ExploreNet is a general-purpose object oriented, 
distributed two-dimensional graphic based computational environment with features to support role- 
playing games for educational purposes, and cooperative learning of many kinds. Among the 
research works, Prada (Prada, Otero, Vala, and Paiva 2004) in his Belief system implements an 
explicit collaborative mode of utilisation. The collaborative mode will challenge learners to share a 
virtual greenhouse. Each group will be able to alter some environmental conditions parameters to 
maximise their crop. Liu focuses on Web-based collaborative virtual learning environment (Liu 2005). 
He provides a cognitive learning architecture based on constructivism. The prototype learning system 
is effective based on the evaluation. Interaction can promote learners’ active learning. During the 
studying experience, interaction can offer the learner various controls, such as interacting with the 
virtual environment and manipulating characters or objects in the virtual environment. VLE 
technologies can address a wide range of interaction capabilities. Picard and her affective computing 
groups (Ahn and Picard 2005), describe affective wearable computers that are with users over long 
periods, like clothing, and that are therefore potentially able to build up long-term models of users’ 
expressions and preferences. The affective wearables offer new ways to augment human abilities 
such as assisting the speaking-impaired and helping remember important information that is 
perceived. 

Synthetic characters with its significant feature of interactive and intelligent behaviour can provide a 
potential tool for learning in VLE. A number of worthy systems and architectures for synthetic 
character behaviour control have been proposed (Bruce, Marc and Yuri 2002). For example, 
Blumberg and his synthetic character groups focus on developing a practical approach for real-time 
learning using synthetic characters. Their implementations are grounded in the techniques of 
reinforcement learning and informed by insights from animal training^ Burke, Isla, Downie, Ivanov, and 
Blumberg 2001). Similarly, USC/ISI developed a pedagogical agent called Steve that supports the 
learning process. Agent Steve can demonstrate skills to students, answer student questions, watch 
the students as they perform tasks, and give advices if students run into difficulties. Multiple Steve 
agents can inhabit a virtual environment, along with multiple students. Therefore, it helps to make it 
possible to train students on team tasks. At the same time, giving synthetic characters with emotions 
and personality has a powerful ability to capture and hold students’ attention. Psychologists and 
pedagogues have pointed out the way that emotions affect learning. According to Piaget (Piaget 
1989), it is incontestable that the affection has an accelerating or perturbing role in learning. A good 
part of the learners who are weak in mathematics, fails due to an affective blockage. Coles (Coles 
2004) suggests that negative emotions can impair learning; and positive emotions can contribute to 
learning achievement. Some educational systems have given attention to generation of emotion in 
pedagogical environments (emotion expression and emotion synthesis) (Lester 1999) and to the 
learner’s emotion recognition (Conati and Zhou 2002), pointing out the richness presented in affective 
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interaction between learner and tutor. We argue that socially appropriate affective behaviours provide 
a new dimension for collaborative learning systems. The system provides an environment in which 
learning takes place through interactions with a coaching computer agent and a co-learner, an 
autonomous agent that makes affective responses. There are two kinds of benefits for learning in the 
collaborative learning environment. One is what is often called ‘learning by teaching,’ in which one 
can learn given knowledge by explaining it to another learner. The other benefit is often called 
‘learning by observation,’ in which one can learn given knowledge by observing another learner 
working on problem solving, teaching other learners, and so on. While in these approaches of 
collaborative learning, are primarily concerned with is knowledge-based, goal-oriented, and rational, 
and thus social intelligence might only be utilised as a side effect. 

Affection in our framework is considered from various angles and on different levels. 

■ The emotional state of the learner will be modelled by an event appraisal system. 

■ The emotional state of the tutor is modelled as well, including values for emotions and parameters 
such as satisfaction, disappointment and surprise. 

■ The dialogue acts come in different forms with variation in affective values. 

■ Various affective parameters are used in determining which tutoring strategy to use and which 
instructional act to perform (sympathising or non-sympathising feedback, motivation, explanation, 
steering, etcetera). 

3. Emotional concept ontology 

A verbal dictionary can be described as a tool that aims to provide a partial solution for the problem 
where two persons neither understand the language the other is speaking but still want to 
communicate. One can just look up the meaning of the words of another language. A nonverbal 
dictionary has the same concept of a verbal dictionary, but it differs in the type of information that is 
stored. Instead of words, a nonverbal dictionary contains information about all the ways people 
communicate with each other nonverbally such as facial expressions in our case to construct the 
emotional ontology. It is well accepted that a common ontology holds the key to fluent communication 
between agents; most researchers believe that the common ontology is domain ontology. This 
assignment can be considered as the extension of a previous work, named FED (an online Facial 
Expression Dictionary) (Jongh 2002) concerning a nonverbal dictionary. Before we define our 
research question and objectives, we summarise the idea of a specific part of FED. We only focus on 
that part of FED, which allows the user to send a picture. This image input will be labelled by 
emotional word (happiness, sad, etc.). FED requires the user to manually locate the face and facial 
characteristic points (FCPs). The FCPs are predefined conform the Kobayashi and Hara face model. 
After manually selecting and submitting the points an emotional word will be output. Thus, FED lacks 
the ability of automatic extraction of facial characteristic points that are needed for the facial 
expression recognition process. In the current situation user interaction is needed to complete the 
whole procedure. In our system the emotional ontology(Cowie, Douglas 2005), cover the major role 
that helps the emotional agents’ to distinguish emotions. These knots represent features of a current 
emotion: for example labels or distances etc. that is the case of APML (Affective Presentation Markup 
Language)(VHML 2001). 

4. Update of emotional markup language (EML) 

What exactly do we mean by "emotion"? There is much disagreement on this, but one of the most 
useful definitions, by psychologist Magda Arnold, draws a careful distinction between states and 
behaviours. In Arnold's theory emotional experience proceeds in three steps: (1) Perception and 
appraisal (external stimulus is perceived and judged good, bad, useful, harmful, etc., mostly based on 
learned associations); (2) Emotion (internal state of arousal or "feeling" arises, involving physiological 
effects); then (3) Action (specific behaviour such as approach, avoidance, attack, or feeding, 
depending on emotional intensity, learned behavioural patterns, and other motivations simultaneously 
present). In this view emotion is an internal state, not a behaviour or a perception of external reality. 
(Robert and Freitas 1984) The Emotion Markup Language (EML) (Marriott 2002) defines the emotion 
elements that affect the VH (virtual human) regarding voice, face, and body. The speech and facial 
animation languages therefore inherit these elements. We have realised some modifications to the 
APML (Affective Presentation Markup Language) (DeCarolis, Carofiglio, Bilvi, and Pelachaud) 
language in order to allow the EECA to communicate a wider variety of facial expressions of emotion 
as well as to allow for a more flexible definition of facial expressions and their corresponding 
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parameters. These modifications refer mainly to facial expressions timings as well as to their intensity; 
intensity corresponds to the amplitude of facial muscles movements. For each APML tag we have 
introduced some new attributes like frequency. The facial expression of an emotion has a limited 
duration (1/2 to 4 seconds), and the facial muscles cannot hold the corresponding expression for hours 
or even minutes without cramping. 

4.1 Attributes of EML 

EML is an XML (W3C 2002) (Extensible Markup Language) compliant text marks up language. This 
implies that it conforms to a standard for the World Wide Web and hence it can be used with 
(sufficiently powerful) web browsers. 

Frequency: the number of times an emotion is felt 

Duration: Specify the time taken in seconds or milliseconds of the emotion existence in the human 
being. 

Intensity: Specify the intensity of this particular emotion, either by a descriptive value or by a numeric 
value. 

Wait: Represent a pause in seconds or milliseconds before continuing with other elements of EML 
(<angry> <disgusted><neutral><surprised> <happy><sadness><fear>) 

4.2 Update of EML elements 

In our framework we propose is the EmotionStyle language, designed to define style in terms of 
multimodal behaviour, and make an EECA display, and recognise emotion accordingly. A new feature 
was added to the EML language. This was to add a distances and frequency attributes to EML in order 
to make to describe more carefully the facial expression. For each we have introduced some distances 
like D1 to D6. 

< Neutral> 

The neutral face represents the reference emotion. The concept of the neutral face is fundamental 
because all the distances describe displacements with respect to the neutral face. 

Description: Facial expression. {D1-D6}=initialised 
Attributes: Default EML attributes. 

Properties: All face muscles are relaxed, the eyelids are tangent to iris, lips are in contact, the mouth is 
closed and the line of the lips is horizontal. 

<Angry> 

Description: Facial expression. {D2 decreases}, {D1 increases}, {D4 either decreases D4 increases} 
Attributes: Default EML attributes. 

Properties: The internal corners of the eyelids decrease together, the eyes are opened largely; the lips 
join each other or they are opened to make the mouth appear. 

< Disgusted > 

Description: Facial expression. {D3 increases AND D4 increases}, {the other distances remain 
constants} 

Attributes: Default EML attributes. 

Properties: The superior lip gets up and is stretched in asymmetrical manner, the eyelids are 
deconstructed. 

< Surprised > 

Description: Facial expression. {D2 increases}, {D1 increases}, {D4 increases}, {the other distances 
remain constants} 

Attributes: Default EML attributes. 

Properties: the eyelids gets up, the mouth is opened. 

< Happy > 

Description: Facial expression. {D4 increases}, {D3 decreases and D6 decreases}, {the other distances 
remain constants} 

Attributes: Default EML attributes. 

Properties: (the mouth is open), the commission are stretched back to the ears, the eyelids are 
stretched. The eyelids gets up, the mouth is opened. 

< Sadness > 

Description: Facial expression. {D2 increases and D5 decreases}, {D1 decreases}, {the other distances 
remain constants} 

Attributes: Default EML attributes. 
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Properties: The internal corners of the eyelids to the height; the eyes are closed slightly; the mouth is 
stretched. 

< Fear> 

Description: Facial expression. {D2 increases and D5 increases but more that D2} 

Attributes: Default EML attributes. 

Properties: the eyelids get up together and their internal parts go to the height. The eyes are contracted 
and in alert. 

4.3 Temporal facial expression features 

The facial expression can be defined in relation with the time of changes in the facial movement and 
can be described according to these three temporal parameters: 

Duration of Onset: how much time is necessary for the emotion to appear? 

Duration of Apex: how much time the expression remains in this position? 

Duration of Offset: how much time so that the expression will disappear? 

5. EMASPEL framework 

5.1 Architecture 

The figurel illustrates the architecture of a peer in our P2P e-learning system. In order to promote a 
more dynamic and flexible communication between the learner and the system, we integrate five kind 
of agent: 



Peer 


Figure 1: EMASPEL architecture 


5. 1.1 Interface agent 

■ Transmit the facial information coming from the learner to the other agents of the Multi-Agents 
System (MAS). 

■ Assign the achieved actions and information communicated by the learner, to agents Curriculum, 
EECA and the other agents of the MAS. 

5. 1.2 Emotional embodied conversational agent 
5.1.2.1 Motivation 

Agents cannot be content to be intelligent, but must be endowed also by emotions and personality. In 
the same way the communication in natural language is not enough, it must be doubled by nonverbal 
communication. Agents are able to give natural responses and therefore come across as believable 
and even interesting conversational partners. (Prendinger, Ishisuka 2004) Animated pedagogical 
agents are “lifelike autonomous characters that cohabit in learning environments with learners to 
create rich, face-to-face learning interactions”. Animated agents carry a personal effect, which is the 
presence of a lifelike character that can strongly influence learners to perceive their learning 
experiences positively (Alexander, Sarrafzadeh and Hill). It is widely accepted that animated agents 
capable of emotion expression are crucial to make the interaction with them more enjoyable and 
compelling for users (e.g. Lester, Towns, and FitzGerald 1999). Emotional behaviour can be 
conveyed through various channels, such as facial display (expression). The so-called ‘basic 
emotions’ approach (Ekman 1999) distills those emotions that have distinctive (facial) expressions 
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associated with them and seems to be universal: fear, anger, sadness, happiness, disgust and 
surprise. More accurately, Ekman prefers to talk about (basic) emotion families. Thus, it consists to 
have many expressions for the same basic emotion. The characteristics of basic emotions include 
quick onset (emotions begin quickly) and brief duration, which clearly distinguish them from other 
affective phenomena such as moods, personality traits or attitudes. Note that only enjoyment and 
possibly surprise are ‘positive’ emotions. The enjoyment family covers amusement, satisfaction, 
sensory leisure, pride, thrill of excitement and contentment. Interestingly, the positive emotions do not 
have a distinct physiological signal. Ekman explains this by referring to the minor relevance of positive 
emotions in evolution. 

5.1.2.2 Description 

In the construction of embodied agents capable of expressive and communicative behaviours in the 
e-learning environment, an important factor is the ability to provide modalities and conversational 
facial expressions on synthetic faces. For example, an animated interface agent is now being used in 
a wide range of application areas including personal assistants, e-commerce, entertainment and e- 
learning environments. Amongst our objectives, is to create a model to generate and visualise 
emotions on embodied conversational agent. The emotions are particularly important for a 
conversational agent since they reveal an essential share of the speech through nonverbal signals. 
William James perceives the emotions like a direct response to the perception of an event 
contributing to the survival of the individual and insists on the changes induced on the body behaviour 
of the individual. The body answers initially in a programmed way of this change; constitutes so what 
one calls the emotions. The feedbacks of the body by the nervous system contribute largely to the 
experiment of the emotions. Research proved that the emotions succeed the facial expressions. 
During the learning process and when interacting with the learner, some tutoring agents may want to 
express affects. Thus, they use EECA, which is able, within a specific activity to translate through a 
character the emotions of the tutoring agent. It has to be aware of the concerned task and of the 
desired emotional reaction (by the designer or the concerned tutoring agent). The emotional state of 
EECA is a short-term memory, which represents the current emotional reaction. To be able to 
compute emotion, a computational model of emotion is required. Our approach was built on Fridja 
model. (Frijda 1986) 

5.1.2.3 Design of EECAs 

This sub-section is about the design of Emotional Embodied Conversational Agents (EECAs). In this 
field of Human Computer Interaction (HCI) and Artificial Intelligence (Al), the design of EECAs or 
‘virtual humans’ and the communication between those agents and human users is the main object of 
research. A lot of effort was put into making ECAs more natural, believable, and making 
communication with ECAs more affective, efficient, and fun. One way to improve communication in 
this way is to make the agent more actively involved in building a relationship with the user. An agent 
that is able to observe the user, and with its personality, appearance, and behaviour is able to 
respond to the (implicit) likes and dislikes of the user, in such a way that it can become acquaintances 
with the user and create an affective interpersonal relationship. Such an agent could have additional 
benefits over a ‘normal’ Embodied Conversational Agent in areas such as e learning. EECA is made 
of three layers (modules) (figure.2):_the first one (perception layer) captures and extracts the facial 
expressions (image acquisition and face tracking) and proceeds to its categorisation (classification). 

The second one (cognition layer) analyses and diagnoses the perceived learner’s emotional state. 
The third one (action layer) takes decision on remedy pedagogical actions to carry out in response to 
the actual emotional state. Tutoring agents are then informed and may access information in the new 
affective state to adapt the current tutoring flow accordingly. The cognitive layer includes two main 
processes named analysis and diagnosis. The analysis of an emotional state recognised by the 
perception layer makes it possible to translate the meaning of this emotion in the learning context. It is 
carried out by taking into account several elements including the recognised emotion, the current 
affective profile, the historic of the actions realised before the emotion expression, the cognitive state 
of the learner, the emotion evolution and the social context (if it corresponds to social or a 
collaborative learning). 
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Figure 2: The emotional embodied conversational agent (EECA) Architecture 


Agents are virtual human beings. They are designed to imitate or model human Behaviour. Human 
Behaviour is complex and many-sided. Nevertheless it is possible to argue that human Behaviour can 
within limits be modelled and can thus be made comprehensible and predictable. Physical, emotional, 
cognitive, and social factors occur in all forms of human behaviour. Approaches, which regard human 
beings exclusively as rational decision makers, are of limited value. The modelling of human 
Behaviour plays an important role in all areas in which action planning, decision-making, social 
interacting and suchlike play a part. These areas include in particular: Sociology Teaching and 
education. Consequently, the internal state of the EECA agent is based on the PECS architecture 
proposed by Schmidt (2000). The PECS architecture is a model of agent that aims to simulating the 
human behaviour in a group. PECS stands for Physical Conditions, Emotional State, Cognitive 
Capabilities and Social Status. These are the four main building blocks of a particular PECS agent 
architecture adding a Sensor-Perception module and a Actor-Actor module, (figure.2) The PECS 
reference model aims at replacing the so-called BDI (Belief, Desire and Intention) architecture. (Rao 
1995) Architectures, such as BDI, which conceive of human beings as rational decision makers, are 
sensible and useful to a very limited degree only. Restriction to the factors of belief, desire and 
intention is simply not appropriate for sophisticated models of real systems where human factors play 
an important role. PECS’s agent model consists of three horizontal layers: 


1. Information Input Layer: This layer processes the input taken from the agent environment and 
consists of two components: The Sensor and Perception components. The sensor component takes 
the external data by means of a set of sensors and the Perception component filters the no relevant 
data and processes the information. The perceptions are used to update the mental state of the agent 
or for learning purposes. 

2 . Internal Components Layer: The personality of the agent is modelled at the second layer. Thus, 
the parameters of this second layer are crucial to find out the response of the agent to the input taken 
by the information layer. They consist of four components: Physics, Cognitive, Emotional and Social 
Status. The physical and material properties of the agents are modelled in the Physical component. 
The emotions that can affect the behaviour of the agent are modelled as part of the Emotional 
component. The agent’s experience and knowledge are part of the Cognitive component; and finally, 
the social features of the agent (e.g., whether the agents like to socialise or they prefer to be alone) 
are described in the Social Status component. 


3. Agent Output Layer: This layer is in charge of modelling the set of possible actions and the 
selection process, and thus it produces the response of the agent and consists of two components: 
The Behaviour and Actor components. The Behaviour Component selects the action(s) that are 
associated with the input information that reaches the agent and the agent’s response is based on its 
internal parameters. The actor component takes the action and executes them. The PECS 
architecture is not based on any social or psychological theory. The architecture is mainly an 
integrated model in which several fundamental aspects to human behaviour and decision-making 
process are taken into account. (Miranda and Aldea 2005). The purpose of the emotional agents 
consists at extracting the facial expressions (acquisition and facial alignment) and subsequently 
categorising them using the temporal evolution of distances D, like it is demonstrated in table 1 .The 
analysis of table 1 shows that it will be possible to differentiate between different emotions while being 
interested in priority in the D, distances which undertake significant modifications. Indeed, there is 
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always at least one different evolution in each scenario. The EECA first carries out an analysis of the 
emotional state of the learner. The purpose of this analysis is to translate the meaning of the emotion 
in the training context. It is achieved based on several factors such as: the emotion sent by the 
emotional agents, the current emotional profile, the history of the actions carried out before the 
appearance of the emotion, the cognitive state, the evolution of the emotion and the social context (if 
it is about a social training or collaborative). 

The expressions in input are “joy”, “fear”, “dislike”, “sadness”, “anger”, “surprised” and the analysis 
make it possible to conclude if the learner is in state of “satisfaction”, “confidence”, “surprise”, 
“confusion” or “frustration”. The interpretation of the analysed emotional state is then established. It 
will consequently determine the causes having led to this situation (success/failure with an exercise, 
difficulty of work, misses knowledge, etc), while being based again on the cognitive state of the 
learner and thus making it possible to the tutor to take, if it is necessary, the suitable teaching actions. 
The role of the action layer is to define, even if necessary, a whole of tasks that allows to remedy at 
the observed emotional state; in order to bring the learner in a state more propitious to the 
knowledge's assimilation, (figure.2). For this reason a collaborative reciprocal strategy in ITS can gain 
advantage from "mirroring" and then assessing emotions in P2P Learning situations. 

5.1.3 The emotional agents 

Integrated into a learning environment, aim at capturing and managing the emotions expressed by the 
learner during a learning session. They currently capture emotions only through facial expression 
analysis and they are in charge of learner emotion detection. They recognised the learner emotional 
state by capturing emotions that he or she expressed during learning activities. (Nkambou 2006). For 
making the affective communication between an EECA and a learner, they need to be able to identify 
the other’s emotion state through the other’s expression and we call this task emotion identification 
established by the emotional agents. Extracting and validating emotional cues through analysis of 
users’ facial expressions is with high importance for improving the level of interaction in man machine 
communication systems. Extraction of appropriate facial features and consequent recognition of the 
user’s emotional state is the topic of these emotional agents. 

■ Analysis of facial expression. The analysis of the facial expressions by the emotional agents is 
generally done according to the following stages: detection of the face, the automatic extraction of 
contours of the permanent features of the face: the eyes, the eyebrows, and the lips. Extracted 
contours being sufficiently realistic, we then use them in a system of recognition of the six 
universal emotions on the face. 

■ Recognition and interpretation of facial expression 



Figure 3: Definition of the distances Dj 

The Classification is based on the analysis of the distances computed on face’s skeletons. The 
distances considered make it possible to develop an expert system (for classification), which is 
compatible with the description MPEG-4 of the six universal emotions. Contours of the eyes, the 
eyebrows and the mouth are extracted automatically by using the algorithms described in (NEJI, Ben 
Ammar and Alimi 2004, BEN AMMAR, Neji and Alimi 2005). The segmentation leads to obtain what 
we call skeleton of expression. Six distances were defined: D1: opening of the eye, D2: outdistance 
between the interior corner of the eye and the eyebrow, D3: opening of the mouth in width, D4: 
opening of the mouth in height, D5: outdistance between the eye and eyebrow and D6: outdistance 
between the corner of the mouth and the external corner of the eye (cf Figure 3). 

■ Joy: {D4 increases}, {D3 decreases and D6 decreases}, {the other distances remain constant} 

■ Sadness: {D2 increases and D5 decreases}, {D1 decreases}, {the other distances remain 
constant} 
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■ Anger: {D2 decreases}, {D1 increases}, {D4 either decrease D4 increases} 

■ Fear: {D2 increases and D5 increases but more that D2} 

■ Disgust: {D3 increases AND D4 increases}, {the other distances remain constant} 

■ Surprised: {D2 increases}, {D1 increases}, {D4 increase}, {the other distances remain constant} 

The table 1 gives a scripts of evolution of the distance Di for the six emotions (tmeans increase ,i 
means decrease and " = " translates the absence of evolution). Notice that for the fear, we do not 
make any hypothesis on the evolution of DI because we do not know how to translate the condition 
{eyes are contracted and in state of alert}. 

Table 1: Di evolution for every emotion 
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The classification of an emotion is based on the temporal evolution of the information contained in the 
"skeleton" resulting from this stage of segmentation (temporal evolution of six characteristic distances). 
For example, joy and disgust differ by the evolution of the distance D6. One notes that emotions (joy 
and surprise) differ by the evolution of distances DI, D2, D3 and D6. This permits a distinction between 
these two emotions. 

5.1.4 Curriculum agent 

Saves the history of progression of the learner in the exercise. While analysing the profile of the 
learner, this agent proposes sessions of activities subsequently to apply. 

The agent curriculum keeps the trace of: 

■ The evolution of the interacting system with learner 

■ The history of progression of learner in the exercise. 

■ The agent curriculum carries out the following operations: 

■ To manage the model of learner throughout the training. 

■ To initialise the session of training by communicating the exercise to the learners according to their 
courses. 

■ To be the person in charge for the individualisation of the training. 

■ To carry out the update of the history of the learner model. 

■ To record in the base of errors the gaps met (errors made by learner) to help the tutor to be useful 
itself of this base to direct its interventions. 

5.1.5 Tutoring agent 
The tutor's role is: 

■ To ensure the follow-up of the training of each learner. 

■ To support learners in their activities. 

■ To support the human relations and the contacts between learners. 

■ To seek to reinforce the intrinsic motivation of learner through its own implication from guide who 
shares the same objective. These interventions aim at the engagement and the persistence of 
learner in the realisation from its training. 

■ To explain the method of training and to help to exceed the encountered difficulties. 

■ To help the learner how he can evaluate his way, his needs, his difficulties, his rhythm and his 
preferences. 
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5.2 Implementation 


5.2.1 The Interaction among agents 

The interaction among human agents is not restricted to the proposed computational model. On the 
contrary, the computational interaction among the artificial agents aims at contributing even more for 
the communication and the exchange among the human agents. The interaction will be one of the 
main objectives of this model, because the proposal is about a model of collaborative learning. The 
several interaction forms involved in the model are interaction among artificial agents; interaction 
among artificial and human agents, and interaction among human agents. In respect to communication 
among the human agents, the system offers tools (synchronous or asynchronous) when physical 
presence is not possible (for example, in the case of virtual classes). 

5.2.2 The organisational model 

Our organisational model is based on the Agent, Group and Role Meta Model (AGR for short) (Ferber 
and Gutknecht 2003). This Meta Model is one of the frameworks proposed to define the organisational 
dimension of a multi-agent system, and it is well appropriate to the e-learning context. According to this 
model, the organisation of the system is defined as a set of related groups, agents, and roles. There 
are several reasons, which justify the importance of this Meta Model. The main reasons are the 
following: (i) it is possible to construct secure systems using groups viewed as “black boxes” because 
what happens in a group cannot be seen from agents that do not belong to that group, (ii) it is possible 
to construct dynamically components of system when we view system as an organisation where 
agents are components. Adding a new group or playing a new role may be seen as a plug-in process 
where a component is integrated into a system, (iii) Semantic interoperability may be guaranteed using 
roles because a role describes the constraints (obligations, requirements and skills) that an agent will 
have to satisfy. 


5.2.3 Implementation 


We programmed agents used in the EMASPEL Framework (figure 4) with the MadKit (Ferber and 
Gutknecht 1998) Platform. MadKit is a modular and scalable multi-agents platform written in Java and 
built upon the AGR (Agent/Group/Role) organisational model: agents are situated in groups and play 
roles. MadKit allows high heterogeneity in agent architectures and communication languages and 
various customisations. In fact, MadKit does not enforce any consideration about the internal structure 
of agents, thus allowing to a developer to freely implement his own agent architecture. Communication 
among agents is implemented by a set of communication primitives, which is a subset of FIPA-ACL 
(FIPA 2004), extended with specific primitives. We used the JXTA Framework (Gong 2002) to build an 
open source p2p network. 



Figure 4 : EMASPEL Framework 


6. Conclusions and further work 

Emotion analysis may reveal if the learner feels “satisfaction”, “confidence”, “surprise”, “confusion”, or 
“frustration”. These states are more precise in an educational context and appropriate pedagogical 
actions can be taken in order to influence those emotions. Another important process is the diagnosis 
of the analysed emotional state. This process determines the possible causes which has led to this 
situation (success/failure in an exercise, difficulty of the tasks, lack of knowledge, incorrect command of 
the knowledge, etc.). This is done using the learner’s cognitive state and the history of his actions. 
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Showing emotions, empathy and understanding through facial expressions and body language is 
central to human interaction. More recently, emotions have also been linked closely with decision¬ 
making, problem solving and intelligence in general. We therefore argue that computer-based 
communication technologies ought to emulate this in some way. We have conducted an experimental 
study on visualisation and recognition of emotion in the human face and an animated face. The study 
used six "universal" facial expressions of emotion, as established by Ekman: happiness, surprise, 
anger, fear, sadness, disgust, together with the neutral expression. Results show that emotions can be 
visualised with a limited number of facial features, and build a potentially strong basis for 
communication in collaborative environments. To further establish the possible role emotions can play 
in collaborative environments, we are currently concentrating on real-time interaction. A group of 
people enters the virtual space and is assigned a task to complete. 

The main objective of the experiment is to investigate how the perception of emotion affects individual 
and group experience in the virtual world. From the real world, we know that emotions of others 
influence us in our decisions and in our own emotional state. Emotions can motivate, encourage, can 
help us achieve things. But they can also change our mood to the negative, make us feel angry or sad 
when someone else feels that way. Emotions are contagious, and their contagious nature in the real 
world is potentially transferable and beneficial to the virtual world. The proposed framework mainly 
includes the peer-to-peer based network platform, for further work we would like to: Integrate the peer- 
to-peer based network platform into grid system. The newly emerged Peer-to-Peer (P2P) and grid 
computing will serve as the key driven forces to bring revolutionary impact on the future society. 
Standardise the e-learning materials: we will implement the SCORM (ADL 2004) specification to 
describe the educational resources in EMASPEL, which will provide the interoperability with other 
system as well as the reusability of learning materials. 
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